Repository navigation
WO20: tiny-student bridge arm verdict + static-int8 sweep verdicts (issue #441) - #442
Merged
Merged
Conversation
added 7 commits
November 16, 2026 21:43
…tool Campaign A: the microkimi-recipe pre-registration (TODO.sota-2026/06) — scan-first directional ridge-R2 pairing with the 0.30 / 2x-scrambled-control kill rule, base recipe = the 2.0 rung verbatim, judge = paired bootstrap vs r7 on the full 1,200-paragraph windowed protocol, tolerance teacher+0.5pp. Campaign B: tools/wo20_static_int8_sweep.py — six released fp32 artifacts through the shipped build_int8_static_zip path; judge = WO03 int8 CER gate (<=2.0pp) + confident flips (<=1%) against the fp32 zip under one framing, plus kit/fixture rows, size and CPU/CUDA latency. Eval slices under conformance/wo20/ with full provenance (400-row eval slices only; never training data).
…ries
load_slice read {name}_eval_slice.jsonl but the committed fixtures are
{name}_eval_400.jsonl; three of the six targets (tha-base, heb-1.0,
heb-1.1) pin split parts with per-part digests plus a whole-file digest —
fetch both layouts, verify every digest.
onnxruntime-gpu 1.23 cannot load the CUDA EP in the job image (CUDA 13 libs missing) and silently fell back - so every measurement was already CPU-framed; drop the misleading cuda variants and declare the framing. The tha-base arm died from holding four 2.8GB graph sets resident: phase the sweep (fp32 decode + latency -> free; build -> free; int8 decode + latency -> free; margins last) with gc between phases.
…ep 2 PASS / 3 FAIL / 1 N/A Campaign A (r7 -> tiny student, microkimi recipe): the scan-first frozen bridge arm at the matched 3-epoch rung scores 4.0545 full-set DER-CE - GATE-FAIL vs teacher+0.5pp (2.7093) but SEPARATED-IMPROVEMENT over the published frontier: paired delta vs G2a -0.4385pp CI95 [-0.65, -0.28] p=0 at half G2a's epochs. First separated-positive student-side lever on record; pre-registered in TODO.sota-2026/06 with the result recorded. Campaign B (static-int8 sweep): urd-vocalized-1.0 PASS (CER-delta 0.0902pp, zero confident flips, x0.36 size, x1.91 CPU) and heb-vocalized-1.1 PASS (0.2907pp, 0.12%); tha-base, heb-1.0, heb-small FAIL (two distinct failure mechanisms - margins-dominated and CER-dominated); tha-phonemic-1.0 N/A (plain-decoder artifact, recipe is KV-only). Failing models keep their fp32 artifacts; nothing published; owner decisions held on publication.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
e70ce991…, 10,995 steps over four resumable 4h windows) scores 4.0545 full-set DER-CE. Strict gate (teacher+0.5pp) FAILS — but this is a SEPARATED-IMPROVEMENT over the published frontier: paired Δ vs G2a −0.4385pp, CI95 [−0.65, −0.28], p=0 (G2a's released predictions re-scored), at half G2a's epochs. First separated-positive student-side lever on record.isx-training:bucket:wo20-tiny-g2w/(predictions, final_eval.json),wo20-quant/(per-model reports, candidate fixtures, built int8static variants),wo20-tiny-scan/(scan.json, bridges.npz).Test plan
wo20-ml-tiny) and the banked-prediction scoring path